A tool for the synchronization of speech and mouth shapes: LIPS
نویسندگان
چکیده
This paper presents a new approach to improve the phonemebased lipsync process. The lipsync process is a module in the animation production pipelines of 2D and 3D cartoons. It consists in generating the mouth positions of a cartoon character from dialogue recorded by an actor. The result is a sequence of time markers which indicate the series of mouth shapes to be drawn. We propose to speed up the lipsync process using tools coming from the field of automatic speech recognition. We describe the LIPS tool (LIPS Logiciel Interactif de PostSynchronisation : lipsync Interactive Software) and the generation of the acoustic models required by the tool and we present results obtained on two cartoons. Using the LIPS tool for the post-synchronization of 52 minutes of cartoon reduced the step of post-synchronization from 14 days to 3 days.
منابع مشابه
Oral Motor Indexes in Iranian 4- to 9-month-old infants: A short report
Introduction: Examination of oral motor indices in infants helps treatment team members identify infants who are delayed in these indices earlier and use the order of occurrence of these indices in normal infants as a criterion for treating infants with motor delay. Literature shows that oral movement indices are different in different races, cultures, and languages. The aim of this study was t...
متن کامل3. Talking Agents: the Same Lip Model We Use 5 Speech-assisted Video Coding 3 Image Analysis: Locating Feature Positions in Face 3.1 Finding Face in Image Speech-assisted Video Processing: Interpolation and Low-bitrate Coding
for interpolation can be used to generate human-like talking agents for human-machine interface, computer-aided instruction, video games, and animation. For example, we can take a still picture of a real person, t the lip model on the mouth, and deform the mouth to produce lip-synchronized talking head. Moreover, if we create a table for gestures used in sign language, and a model for arms and ...
متن کاملLip synchronization of speech
Lip synchronization is the determination of the motion of the mouth and tongue during speech. It can be deduced from the speech signal without phonemic analysis, and irrespective of the content of the speech. Our method is based on the observation that the position of the mouth over a short interval of time can be correlated with the basic shape of the spectrum of the speech over that same inte...
متن کاملA symbolic system for multi-purpose description of the mouth shapes
A multi-purpose symbolic system which can describe all of the mouth shapes conveying various kinds of linguistic information in signing, speech dialogue, and speechreading is developed, taking account of the constraints of anatomical structure and neural control and the possibility of visual discrimination. In order to make the changes in the three-dimensional shape of lips, the changes are des...
متن کاملA Lip Localization Based Visual Feature Extraction Method
This paper presents a lip localization based visual feature extraction method to segment lip region from image or video in real time. Lip localization and tracking is useful in many applications such as lip reading, lip synchronization, visual speech recognition, facial animation etc. To synchronize lip movements with input audio we need to first segment lip region from input image or video fra...
متن کامل